[Runtime][Vulkan] Add RGP support to TVM for vulkan device #10953

avulisha · 2022-04-10T11:12:06Z

RGP(Raedon GPU Profiler) is a tool used to analyze the applications
run on AMD GPU. RGP captures the data based on VKPresent and provides
the hardware specific information. Allowing the developer to optimize
the application. To add RGP support to TVM, debug labels "AmdFrameBegin"
and "AmdFrameEnd" need to be inserted into the vulkan queue.These Labels
helps the RGP tool to understand the start|end of frame when no present
is available. Thus enabling the RGP tool to capture and analyze the data.

At runtime, set the envirnoment variable "TVM_USE_AMD_RGP=1" to start
inserting the Debug Labels into the vulkan queue.

Signed-off-by: Wilkin Chau Wing-Ki.ChauWilkin@amd.com
Signed-off-by: Anurag Kumar Vulisha AnuragKumar.Vulisha@amd.com

Thanks for contributing to TVM! Please refer to guideline https://tvm.apache.org/docs/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from Reviewers by @ them in the pull request thread.

masahi

Thanks @avulisha! I'll try get this working on my RX6600xt.

masahi · 2022-04-10T21:21:49Z

src/runtime/vulkan/vulkan_instance.cc

@@ -59,6 +59,14 @@ VulkanInstance::VulkanInstance() {
    std::vector<const char*> required_extensions{};
    std::vector<const char*> optional_extensions{"VK_KHR_get_physical_device_properties2"};

+    // Check if RGP support is needed. If needed, enable VK_EXT_debug_utils extension for
+    // inserting debug labels into the queue.
+    const char* val = getenv("TVM_USE_AMD_RGP");


Use BoolEnvironmentVar

tvm/src/runtime/vulkan/vulkan_instance.cc

Line 36 in ef7143e

if (support::BoolEnvironmentVar("TVM_VULKAN_ENABLE_VALIDATION_LAYERS")) {

Use BoolEnvironmentVar

tvm/src/runtime/vulkan/vulkan_instance.cc

Line 36 in ef7143e

if (support::BoolEnvironmentVar("TVM_VULKAN_ENABLE_VALIDATION_LAYERS")) {

Hi @masahi,
Thanks for your time in reviewing the changes. Will Implement your suggestion.
Thanks,
Anurag

masahi · 2022-04-10T21:23:45Z

src/runtime/vulkan/vulkan_stream.cc

@@ -55,11 +55,15 @@ VulkanStream::VulkanStream(const VulkanDevice* device)
  cb_begin.flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT;
  cb_begin.pInheritanceInfo = 0;
  VULKAN_CALL(vkBeginCommandBuffer(state_->cmd_buffer_, &cb_begin));
+
+  profiler_ = new AmdRgpProfiler(device_);


Please do this only when RGP is enabled.

Please do this only when RGP is enabled.

Okay. Will Implement your suggestion.

src/runtime/vulkan/vulkan_stream.h

src/runtime/vulkan/vulkan_wrapped_func.cc

RGP(Raedon GPU Profiler) is a tool used to analyze the applications run on AMD GPU. RGP captures the data based on VKPresent and provides the hardware specific information. Allowing the developer to optimize the application. To add RGP support to TVM, debug labels "AmdFrameBegin" and "AmdFrameEnd" need to be inserted into the vulkan queue.These Labels helps the RGP tool to understand the start|end of frame when no present is available. Thus enabling the RGP tool to capture and analyze the data. At runtime, set the envirnoment variable "TVM_USE_AMD_RGP=1" to start inserting the Debug Labels into the vulkan queue. Signed-off-by: Wilkin Chau <Wing-Ki.ChauWilkin@amd.com> Signed-off-by: Anurag Kumar Vulisha <AnuragKumar.Vulisha@amd.com>

avulisha · 2022-04-11T18:45:06Z

Hi @masahi,
Thanks for your time in reviewing the changes. I have pushed the changes that you have suggested.
Best Regards,
Anurag

masahi · 2022-04-12T23:25:39Z

Hi @avulisha (cc @mei-ye), I want to try this. What is your typical workflow? For example, I want to capture the trace from running https://github.com/apache/tvm/blob/main/apps/topi_recipe/gemm/cuda_gemm_square.py.

It looks like I need to press "Capture profile" button in the profiler UI, but the script quickly finishes before I am able to start capturing. So I'm wondering how you typically workaround that issue. I do see tvm/src/runtime/vulkan/vulkan_instance.cc:65: Push VK_EXT_debug_utils logged.

mei-ye · 2022-04-13T05:06:14Z

To successfully capture a trace, it requires at least five complete Present events. Since the inference time is very short (in 10s of ms), a loop with many iterations is normally required to ensure that the capture is completed before the process is terminated.

avulisha · 2022-04-13T05:11:49Z

Hi @avulisha (cc @mei-ye), I want to try this. What is your typical workflow? For example, I want to capture the trace from running https://github.com/apache/tvm/blob/main/apps/topi_recipe/gemm/cuda_gemm_square.py.

It looks like I need to press "Capture profile" button in the profiler UI, but the script quickly finishes before I am able to start capturing. So I'm wondering how you typically workaround that issue. I do see tvm/src/runtime/vulkan/vulkan_instance.cc:65: Push VK_EXT_debug_utils logged.

Hi @masahi ,
As Mei was mentioning, the run is very short for the RGP tool to capture the traces. For testing, we can use the frontend tests to capture the traces. "TVM_FFI=ctypes python -m pytest -v tests/python/frontend/onnx/test_forward.py"
There are many tests that would be run as a part of frontend tests, allowing the RGP tool to capture the traces.
Thanks,
Anurag

masahi

I'll try capturing later.

masahi · 2022-04-13T10:11:08Z

Thanks! @avulisha

avulisha · 2022-04-13T10:47:30Z

Thanks! @avulisha
Hi @masahi.
Thanks for your time in reviewing the changes and merging them.
Best Regards,
Anurag

@yzh119

* main: (527 commits) [hexagon] 'add_hvx' test to explore HVX usage. (apache#10604) [COMMUNITY] @yzh119 -> Reviewer (apache#10993) [Metaschedule] Make custom schedule_rule registration optional (apache#10975) [ONNX] Add imports for BERT contrib operators (apache#10949) sort axes (apache#10985) [Hexagon] Remove HexagonBuffer external constructor and support (apache#10978) [CI] Update GPU image (apache#10992) [Runtime][Vulkan] Add RGP support to TVM for vulkan device (apache#10953) [FIX] resolve int64/32 for AttrStmtNode (apache#10983) [TVMC] Allow output module name to be passed as a command line argument (apache#10962) [ONNX] Add MatMulInteger importer (apache#10450) [COMMUNITY] @guberti -> Reviewer (apache#10976) Support `qnn.conv2d` in FoldExplicitPading (apache#10982) change Hexagon docker version (apache#10981) remove exception handling of autotvm xgboost extract functions (apache#10948) [CUDNN] Add partitioning support for conv2d and log_softmax (apache#10961) [Hexagon][LLVM] Enable/test tensorized Hexagon DMA on 2d transformed layout (apache#10905) [Hexagon] Move aot/graph_executor interactions into launcher (apache#10907) [HEXAGON] Split huge 1D DMA Transfers into smaller transfers with legal sizes. (apache#10971) [CI][DOCKER] Add pytest-lazy-fixture to images (apache#10970) ...

) RGP(Raedon GPU Profiler) is a tool used to analyze the applications run on AMD GPU. RGP captures the data based on VKPresent and provides the hardware specific information. Allowing the developer to optimize the application. To add RGP support to TVM, debug labels "AmdFrameBegin" and "AmdFrameEnd" need to be inserted into the vulkan queue.These Labels helps the RGP tool to understand the start|end of frame when no present is available. Thus enabling the RGP tool to capture and analyze the data. At runtime, set the envirnoment variable "TVM_USE_AMD_RGP=1" to start inserting the Debug Labels into the vulkan queue. Signed-off-by: Wilkin Chau <Wing-Ki.ChauWilkin@amd.com> Signed-off-by: Anurag Kumar Vulisha <AnuragKumar.Vulisha@amd.com> Co-authored-by: avulisha <avulisha@amd.com>

masahi reviewed Apr 10, 2022

View reviewed changes

avulisha force-pushed the rgp_support branch from f18d8d4 to 81286c8 Compare April 11, 2022 18:41

avulisha requested a review from masahi April 11, 2022 18:42

masahi approved these changes Apr 13, 2022

View reviewed changes

masahi merged commit b542724 into apache:main Apr 13, 2022

driazati mentioned this pull request Jul 14, 2022

TVM v0.9.0.rc0 Release Candidate Notes #12102

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Runtime][Vulkan] Add RGP support to TVM for vulkan device #10953

[Runtime][Vulkan] Add RGP support to TVM for vulkan device #10953

avulisha commented Apr 10, 2022

masahi left a comment

masahi Apr 10, 2022

avulisha Apr 11, 2022

masahi Apr 10, 2022

avulisha Apr 11, 2022

avulisha commented Apr 11, 2022

masahi commented Apr 12, 2022 •

edited

Loading

mei-ye commented Apr 13, 2022

avulisha commented Apr 13, 2022

masahi left a comment

masahi commented Apr 13, 2022

avulisha commented Apr 13, 2022

[Runtime][Vulkan] Add RGP support to TVM for vulkan device #10953

[Runtime][Vulkan] Add RGP support to TVM for vulkan device #10953

Conversation

avulisha commented Apr 10, 2022

masahi left a comment

Choose a reason for hiding this comment

masahi Apr 10, 2022

Choose a reason for hiding this comment

avulisha Apr 11, 2022

Choose a reason for hiding this comment

masahi Apr 10, 2022

Choose a reason for hiding this comment

avulisha Apr 11, 2022

Choose a reason for hiding this comment

avulisha commented Apr 11, 2022

masahi commented Apr 12, 2022 • edited Loading

mei-ye commented Apr 13, 2022

avulisha commented Apr 13, 2022

masahi left a comment

Choose a reason for hiding this comment

masahi commented Apr 13, 2022

avulisha commented Apr 13, 2022

masahi commented Apr 12, 2022 •

edited

Loading